Repository navigation
verify: degrade on a rate-limited 429 instead of raising (0.4.1) - #5
Merged
Merged
Conversation
A 429 that carries Retry-After is the edge rate limiter: transient load on the caller's request path, which should fall through to the LLM like a 5xx rather than throw. A 429 without Retry-After is the monthly verify quota running out and still raises, since that needs a decision, not a silent month of fallbacks; other 4xx still raise too. CacheVerifierError gains a retry_after attribute. Prepares for the hosted service switching its rate-limit response from 503 to the documented 429.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 1 of switching the hosted service's rate-limit response from 503 to the 429 that the API docs already promise.
The hosted API returns 429 in two different cases:
Retry-After.app/routers/verify.py): lasts until next month and needs a decision (top up or subscribe). NoRetry-After.Behaviour change for
verify()/verify_batch()Retry-Afternow degrades like a 5xx: it logs a warning and returnsdegraded=True, withapprovedperfail_open(False by default), instead of raising.Retry-After(quota) still raisesCacheVerifierError, as do other 4xx.CacheVerifierErrorgets aretry_afterattribute.Today's 503 from the limiter already degrades, so this release changes nothing until the server switches.
Tests: rate-limited 429 degrades for single and batch calls; quota 429 (no header) still raises with
retry_after is None; 401 still raises. 38 passed, 1 skipped; ruff clean.Version 0.4.1: pyproject,
__version__, CHANGELOG. Publish by creating a GitHub releasev0.4.1, which triggerspublish.yml.